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ABSTRACT 

In response to calls for increased accountability in 
educational institutions, the American Association of Community 
Colleges developed the following 13 core indicators of institutional 
effectiveness? student goal attainment, fall to fall persistence 
rates; degree completion rates; placement rates in the work force; 
employer assessments of students; number and rate of transfers, 
performance after transfer; basic skills students' success in 
subsequent coursework; demonstration of critical literacy and 
citizenship skills; client assessment of programs; responsiveness to 
community needs; and service area participation. The Colorado^ 
Community College and Occupational Education System has identified 
several challenges for implementing these indicators in the state, 
including the following: (1) developing an instrument for determining 

student goals at entry; (2) establishing a longitudinal database to 
track student outcomes; (3) tabulating state employment information; 
(4) developing mechanisms to survey employers and maximize response 
rates; (5) assessing student transfer intentions at matriculation or 
later at the campus level; (6) addressing literacy and citizenship as 
an outcome; (7) developing data sources to address client 
satisfaction and determine opinions about programs and services; (8) 
conducting community needs assessment, focus groups, and surveys to 
determine responsiveness to community needs; and (9) addressing 
participation rates in the service area throug'n attendance records 
and community impact surveys. (TGI) 
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National. State, and Local Challenges for Community C olleges 

The landscape of higher education, especially across community colleges, is 
changing dramatically. Student Right to Know legislation mandating reporting of 
graduation statistics, dramatic changes in institutional eligibility requirements for 
federal financial aid, evolving requirements for accreditation, reporting mandates for 
Perkins vocational funds, changing student populations, and increased scrutiny by 
state governments and the public have created a challenging environment for 
community colleges. Many institutions have responded to this agenda by identifying 
performance indicators and building effectiveness models. Experience shows that 
some of these models are uncomplicated, some are complex. Other inshtutions, 
perhaps out of annoyance, simply ignore effectiveness indicators until an impending 
accreditation review or required state or federal report reawakens interest. 

At the same time, stakeholders in our colleges are continually faced with 
making judgements about institutional effectiveness. Students, community leaders, 
legislators, business and industry officials, faculty, and administrators make choices 
about our institutions on a continual basis. As they make these choices they 
intersperse information from their experience about high quality (effective) 
performance and poor quality (ineffective) performance into the effectiveness 
equation. These individuals may draw their judgements from d/recf experience with a 
college or through information gleaned from a secondary source , i.e., newspapers, 
television advertising, college catalogs, conversations with friends, etc. Human 
nature virtually guarantees that when individuals lack direct information they will not 
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delay passing judgement on a college. Instead, their judgments will come from 
secondary sources which almost always have little or nothing to do with institutional 

quality. 

Community Colleges typically turn to the factors which are easily quantified to 
respond to questions about institutional goals. Historically, these measures have 
included growth in programs, budgets, and enrollments. Enrollment growth is the 
most frequently used indicator of effectiveness, but its use as the primary indicator of 
effectiveness opens the community college to attack by critics who claim that 
quantity, as expressed by enrollment, camouflages a lack of quality. Other 
indicators, besides growth, may better serve the long-term health of the community 
colleges, provided that these indicators account for differences in size, location, 
funding, campus culture, and administrative philosophies. 

Colleges without a framework to address effectiveness also find it difficult to 
argue persuasively in the state and federal policy and budget setting arenas. This 
vacuum has spurred legislators and other policy-makers to create a sweeping, and 
often bewildering and contradictory, array of accountability requirements. In 
Colorado, the state pursues accountability through HB 1187 which requires 
campuses to submit an annual report to the Colorado Commission on Higher 
Education. Cn the federal level, the impact of requirements found under recent 
federal legislation will require each campus to adopt a framework for educating 
internal and external audiences about institutional effectiveness. 



Rftsnonse by the American Associa tion of Community Collg££S 

In 1994, the American Association of Community Colleges published a 
landmark report. Community Colleges: Core Indicat ors Qf Effectiveness.^ This report 
was the result of work by community college executive officers, university professors, 
and higher education officials. Written to provide a sorely needed focus for the 
issues discussed above, this work provides a small group of core, measures which 
are vital to each institution. In this light, the term ‘core* denotes a critical, focused 



‘ American Association of Community Colleges, (1994). ComniunitY Colle g es: Core Indicators 
of Fffectiveness (AACC Special Reports No. 4), AACC: Washington, D.C. 
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group of indicators, reduced from a multiplicity of earlier models, measures, and 
methodologies. At conclusion, characteristics of the AACC core indicators were 
generalizability across institutions, ease and efficiency of use, relevance to the 
community college mission, and significance to multiple customers. 

It is the authors' belief that core indicators can improve institutional 
performance in fundamental ways beyond the act of assessment. These indicators 
are intended to help the institution focus on those matters that are the most indicative 
of and important to student success. Ultimately, the report states that the use of 
core indicators will increase the public's understanding of the community college, 
especially at the policy level, making it more likely that external decision makers will 
hold our institutions accountable on matters that are appropriate and substantive. 



AACC’s 13 CORE INDICATORS OF EFFECTIVENESS 
Student Progress 

► Core Indicator 1 : Student Goal Attainment 

► Core Indicator 2: Persistence (Fall to Fall) 

► Core Indicator 3: Degree Completion Rates 

Career Preparation 

► Core Indicator 4: Placement Rate in the Work Force 

► Core Indicator 5: Employer Assessment of Students 

Transfer Preparation 

»• Core Indicator 6: Number and Rate Who Transfer 

► Core Indicator 7: Performance After Transfer 

Developmental Education 

► Core Indicator 8: Success in Subsequent, Related Coursework 



General Education 

► Core Indicator 9: Demonstration of Critical Literacy Skills 

► Core Indicator 10: Demonstration of Citizensnip Skills 
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Customized Education 

► Core Indicator 1 1 ; Client Assessment of Programs and Services 



Community Development 

► Core Indicator 12: Responsiveness to Community Needs 
- Core Indicator 1 3: Participation Rate in Service Area 



Challenges in Implementing Core Indicators in Cdo riidQ. 

Colleges within the CCCOES system are well poised to develop these 
indicators. Their implementation would require only minor modifications to the ways 
in which data is collected. Required assessment and program review processes have 
developed campus expertise necessary for developing core indicators. Each 
institution annually develops an accountability document for the CCHE. Campuses 
indicate that CCHE's process is helpful for internal purposes, but could be improved 
for internal use by focusing on outcomes which are important to the community 
college. As campuses have become more sophisticated in dealing with required 
accountability and have increased their expertise, the time may be right to migrate to 
a more meaningful structure to assess quality. 

Campuses also have had to respond to new information needs mandated by 
the Carl Perkins Vocational and Technical Education Act. The result has been to 
demonstrate the need to track students in vocational programs from program entry 
through employment. CCCOES staff have provided several applied workshops in 
methodology for establishing a longitudinal database for cohort tracking. In addition, 
these workshops have sought to determine the person on each campus who is 
ultimately responsible for submitting external data reports, including issues of data 
quality. This responsibility has been difficult to pinpoint. 

The CCCOES Student Information System can collect and maintain the student 
information necessary to implement core indicators which deal with student progress 
(Core Indicators 1 through 3). The chief challenge for each institution will be to 
collect meaningful and timely information about student intent as the basis for 
defining cohorts of students for tracking. At present, our SIS system-through the 
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student admissions form-does not adequately identify student intent to permit 
definition of cohorts. Further, no campus in the system is currently updating student 
intent data on a routine basis. 

Other challenges face campuses in implementing the Core Indicator model. 
Following the a description of the measure and data source required for each core 
indicator is a synopsis of the challenges facing CCCOES institutions in implementing 
that particular core indicator. All descriptions, with the exception of {bracketed 
material] and the italicized section accompanying each indicator entitled ‘Colorado 
Challenge' were taken directly from the AACC publication. 



Core indicator 1 : Student Goal Attainment 

Measure: The proportion of students who on leaving a community 

college report that their original goal in attending (or 
subsequent goal decided while enrolled) has been met. 

Data Source(s): Periodic surveys of the student population according to a 

longitudinal design that includes the determination of 
student goal(s) at entry and a follow-up soon after exit to 
determine if this goal has changed and/or it has been 
met. Most desirable is a method whereby student goals 
are assessed each time the student registers. 

Colorado Challenge: Development of an instrument (e.g., application form) for 

trapping student goals at entry, particularly in the 
employment intent and transfer intention areas. A 
parallel challenge is to develop a method whereby 
student intent is updated at each registration period. 



Core Indicator 2: 
Measure: 
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The proportion of an identified fall term entering 
community college student cohort that is still enrolled for 
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at least one credit the following fall term and has not 



Data Source(s); 


completed a degree or certificate, reported each year 
from entry to exit. An 'identified entering student cohort' 
is defined as all students registering for their first credits 
at the community college in a given term; a fall-term 
cohort is typically used for data analysis and reporting. 

The recommended data source is continuously updated 
longitudinal database that tracks an entering community 
college student cohort from entry to exit using data 
elements such as course and program enrollment and 
degrees and certificates awarded (e.g., Ewell, Jones, and 
Parker, 198S^). 


Colorado Challenge: 


Working with Community College Computer Services to 
program the and the structure necessary to establish a 
longitudinal database. Then, assigning of campus 
. oponsibility for updating cohort files based on 
registration surveys, course file information from SIS, 
and the degrees granted file. Also at issue is the 
concern about campus responsibility for data fidelity. 


Core Indicator 3: 


Degree Completion Rates 


Measure: 


The proportion of an identified entering community 
college cohort officially enrolled in a degree program 
that actually completes a degree or certificate, as 
reported at annual intervals. 


Explanation: 


The emergence of such regulations as Student Right-to- 
Know and numerous state reporting requirements has 



‘ Ewell, P.T., Parker, R.P, and Jones, D.P. (1988). Establishing a Longitudinal Student 
Tracking System: An Implementation Handbook . Boulder, CO: National Center for Higher Education 

Management Systems. 
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highlighted this statistic as a performance indicator, and 
it is thus impossible to ignore. Recognizing that 
community college students take as long as seven or 
eight years to earn a degree, the proposed indicator has 
no 'endpoint* time limit. Also, unlike federal and state 
regulatory statistics, this indicator includes all entering 
students-not just those who are full-time, first-time-in- 
college students. While this may deflate reported 
graduation rates, it more fully reflects actual community 
college populations the majority of which are in fact part- 
time attenders. Given the sensitivity of this statistic, it 
should never be presented in isolation; it should be 
reported selectively so that attendance patterns, course- 
taking behaviors, and intention of different student 
groups are apparent. This indicator should always be 
reported in conjunction with persistence rates (Core 
Indicator 2). 

Data Source(s): The recommended data source is a continuously 

updated longitudinal database that tracks an entering 
community college student cohort from entry to exit 
using data elements such as date of initial enrollment, 
degree awarded, and duration of study. Cohorts should 
be constructed so that they are consistent with (but 
more inclusive than) those required for current federal 
and state reporting. 

Colorado Challenge: Same challenge as for #2. Caution in reporting rates so 

that meaningful and accurate information is reported. 
Implementation will provide an accurate count of 
graduation rates-unlike proposed Student Right to Know 
regulations- since a cohort may exist over the course of, 
say, six (6) to ten (10) years. 
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Core indicator 4: 
Measure: 



Data Source(s): 



Colorado Challenge: 



Placement Rate in the Work Force 

The proportion of an identified entering community 
college student cohort achieving a ’marketable skill' (i.e., 
completing at least three occupational/technical courses 
in a particular field of training) who obtain employment in 
a field directly related to this skill within one year of last 
attendance. 

The predominant method used is a follow-up 
questionnaire administered to former students 
periodically by mail [Note: CCCOES strongly encourages 
telephone surveys as a maximal method to ensure 
higher response rates]. If available, state employment 
information (generally available through the state's 
Department of Labor or equivalent) provides a more 
direct method of assessment. Such information is now 
in place in seven states. The obvious limitation of this 
methodology is the fact that many state databases 
[including Colorado] do not contain the required job 
information to adequately ident.y occupation. 

At the system level, to continue to work with the 
Department of Labor to tabulate state employment 
information. However, these data will not account for 
individuals who are self-employed, federal employees, 
nor graduates who have moved to other states. Would 
also require redefinition of ‘program completer" to 
include those who take three (3) occupationally-related 
courses, necessitating-again-identification of student 
intent at initial entry. At the campus level, to continue to 
survey students for information, including satisfaction 
with courses of study, utilizing the most current 
telephone numbers and/or addresses. 
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Core Indicator 5: 
Measure: 

Data Source(s): 



Colorado Challenge: 



Core Indicator 6: 
Measure: 



Employer Assessment of Students 

The proportion of a sample of regional employers in a 
given field indicating that their employees who received 
training at the community college exhibit skills and job 
performance at rates equivalent or superior to all 
employees. 

The recommended method is a periodic cross-sectional 
survey of a sample of area employers using an open- 
ended questionnaire requesting an employer to list the 
specific programs and institutions from which they like to 
recruit or from which they obtain 'their best employees'. 
Because of typically higher response rates, this is 
preferred to the more typical 'two-phase' follow-up 
method in which former students are first contacted to 
identify their employers and the employers then 
surveyed. Multiple attributes of performance should be 
assessed and a Likert-type rating scale of at least five 
points should be used in the questionnaire. If resources 
are available, in-depth interviews with employers are 
recommended. 

To develop mechanisms to survey employers where it is 
not being done and to report the results of such surveys 
where it is being done. Every effort should be made to 
maximize response rates during the first phase. 
Implementation of this indicator This can increase 
employer responsiveness to college initiatives. 



Number and Rate Who Transfer 

The proportion of an identified entering community 
college cohort actively enrolled in a degree program at 
that institution and completing at least twelve semester 
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hours of credit (or equivalent) who enroll within two years 



Data Source(s): 


for at least twelve college level credits in a degree 
program at a four-year college or university. 

The preferred data source for this indicator is actual 
student record data obtained from the transfer 
institution. Available methodologies listed in order of 
des. .ability include; 

• state-level enrollment information that directly 
matches community college and four-year public 
college attenders by means of student 
identification number. 

• electronic transfer of transcript information in a 
defined format (e.g., SPEEDE, AACROA standard, 
North Texas Consortium). 

• aggregate reporting obtained periodically from 
senior institutions. 

• students self-reports obtained through follow-up 
questionnaire administered to former community 
college students. 


Colorado Challenge: 


At the system level, continue to provide campuses with 
annual aggregate data on student transfer and 
performance. At the campus level, to assess student 
transfer intentions at matriculation or thereafter to 
facilitate follow-up and institutional analyses. 


Core Indicator 7: 


Performance After Transfer 


Measure: 


The proportion of regular college-level courses at the 
transfer (receiving) institution completed with a grade of 
"C‘ or better by students who previously attended the 
community college, compared to a parallel proportion 
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obtained for students who began their studies as first- 



Data Source(s): 


time freshmen at the transfer institution. 
Same as Core Indicator 3, 


Colorado Challenge: 


Same as Core Indicator 6. 


Core Indicator 8: 


Success in Subsequent, Related Coursework for 
Basic Skills Students 


Measure: 


The proportion of identified entering student cohort 
assessed as deficient in one or more of the basic skills 
(reading, writing, and computation) who subsequently: a) 
successfully complete developmental work intended to 
remediate this deficiency, and b) within one year 
complete, with a grade of *C or better, their first college- 
level courses requiring the use of this skill. 


Data Source(s): 


The recommended data source is a continuously 
updated longitudinal database that tracks an entering 
community college student cohort from entry to exit 
using data elements such as basic skill proficiency 
levels, course enrollments, and course grades (e,g,, 
Ewell, Jones, and Parker), 


Colorado Challenge: 


To work with Community College Computer Services to 
program a structure for a longitudinal database for 
cohort tracking, including basic skills proficiency levels 
as well as student intent data. To routine!}' report this- 
and other Core Indicator- information. 
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Core Indicator 9: 

Measure: 

Explanation: 



Data Source(s): 



Demonstration of Critical Literacy Skills 

No single statistic satisfies the intent of this indicator. 

Literacy has become a complex social concept, but this 
does not absolve community colleges from 
demonstrating that they develop such skills. Assessing 
literacy requires a multiplicity of methods appropriate to 
the kinds of literacy tasks that students will be facing in 
the community and on the job. Mere data collection is 
not enough. Once the college community agrees that a 
particular set of literacy skills is important as an 
outcome, it must ensure that such skills are: a) taught 
throughout the curriculum in both transfer and 
occupationally related courses and b) regularly embodied 
in local assessments designed by faculty. 

Many methods are appropriate to determine literacy 
skills at both the institutional and programmatic level. 
Among those recommended are: 

• faculty-designed instruments to be administered 
to students as appropriate and modeled on such 
national literacy assessments as the National 
Assessment of Educational Progress (NAEP) Adult 
Literacy Scales. Typically, these assessments 
present students with natural literacy tasks such 
as reading a set of instructions, interpreting 
published charts and graphs, completing a 
required form, or reading and interpreting a 
schedule. 

• specially designed 'in-baskef exercises that 
require students to assimilate a variety of 
material, interpret its significance, and prioritize it 
for action. Exercises of this kind can be 
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Colorado Challenge: 

Core Indicator 10: 

Measure: 

Explanation: 

Data Source(s): 



embedded in capstone experiences that are a 
part of many applied programs. 

• exercise that require students to read and 
interpret a typical news story. 

• specially designed questionnaire items for 
inclusion on student follow-up surveys about 
behavior in areas such as current reading habits, 
perceptions of current events, etc. 

Address literacy as a community college outcome and 
develop agreement on its parameters and ways to 
measure it 



Demonstration of Citizenship Skills 

No single statistic satisfies the intent of this indicator. 

Citizenship is also a complex attribute that requires the 
use of multiple evidence-gathering techniques to assess 
adequately, and requires an institution-wide process to 
ensure that its development and assessment are 
included in instructional delivery. 

Among the most promising methods that can be used in 
gathering information consistent with this indicator are: 

• student portfolios documenting citizenship 
activities they have participated in while enrolled, 
such as volunteer work, memberships in student 
groups, active participation in political campaigns, 
etc. 

• documentation of student participation in group or 
problem-solving exercises in which leadership and 
group-membership skills are potentially 
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Colorado Challenge: 

Core indicator 11: 
Measure: 

Explanation: 



demonstrated, along with appropriate 
occupational or technical skills. 

» specially designed exercise included in regular 
course examinations that require students to 
determine the impact of their actions on others or 
their broader responsibilities to others. 

• specially designed questionnaire items included in 
alumni follow-up surveys that determine whether 
the student has voted in a local election, 
participated in voluntary or community service 
work, or follows the news regularly. 

Address citizenship as a community college outcome 
and develop agreement on its parameters and ways to 
measure it 



Client Assessment of Programs and Services 

No single statistic currently meets the intent of this 
indicator. 

Responding to the intent of this indicator requires the 
institution to determine: a): the array of different clients 
that it serves; b); the nature, types, and volume of 
programs, services, and activities it provides to different 
clients; and c) the extent to which clients are involved 
and satisfied with programs, services, and activities. 
’Client' is a term that describes many different 
individuals, groups, and organizations coming into 
contact with community colleges for many reasons 
whose involvement and satisfaction must be 
independently determined and assessed. Successful 
reporting of this indicator will thereby require multiple 
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Data Source(s): 



Colorado Challenge: 
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methods for gathering information, including surveys, 
focus group meeting, face-to-face and telephone 
interviews, and participant observation. 

Some of the data sources that can be used to collect 
information consistent with this indicator are the 
following: 

• specially designed evaluations administered to 
clients participating in programs and services to 
determine their expectations and the extent to 
which the program or service met, exceeded, or 
fell short of expectations. 

• periodic surveys carried out with client groups; 
these are typically mail or telephone surveys 
using an instrument designed to collect 
information about both client involvement and 
satisfaction with the program or service. 

• written reports by professional staff members 
based on: a) observation of clients participating 
in programs and services and resulting 
expressions of satisfaction and b) interviews with 
clients during and after contact with the program 
or service 

• focus group meetings with client groups to 
determine involvement and satisfaction with 
programs and services. 

To continue to encourage campuses to address 
client satisfaction and opinions about programs 
and services. 
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Core indicator 12: 
Pleasure: 



Data Source(s): 



Responsiveness to Community Needs 

No single statistic currently meets the intent of this 
indicator. 

Some of the data sources that can be used to collect 
information consistent with this indicator are the 
following: 

• periodic needs assessments carried out with 
citizens in the college's service region; these are 
typically mailed or telephone household surveys, 
using an instrument designed to collect 
information about both current educational and 
service needs, and about contact and satisfaction 
with the college. 

• surveys and/or focus group meeting with 
community organizations, citizen's groups, 
employers, etc. to determine needs and current 
levels of satisfaction. 

• environmental scanning processes designed to 
systematically examine the content of printed 
matter such as newspaper stories, editorials, job 
advertisements, and employment and other 
socioeconomic data about community trends, to 
determine long-term patterns of community 
development and future needs. 

• specially designed evaluations periodically 
administered to participants in each program or 
event sponsored by the college, to assess the 
expectations of those participating and the 
degree to which their expectations were met. 
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Colorado Challenge: To encourage and refine present efforts on the campus 





level 


Core indicator 13: 


Participation Rate in Service Area 


Measure: 


The proportion of the total population aged 17 or over 
residing in the designated service area of the community 
college, participating in at least one organized activity 
(course, program, service, event, etc.) sponsored by the 
college in the past year. 


Data Source(s): 


The most common method is attendance records at all 
college-sponsored classes, events, activities, etc. 
maintained by the college. The recommended method is 
a periodic community impact survey administered to a 
representative sample of residents in the service region. 
This survey should contain items describing various 
types of contact with the college (formal and informal), 
and overall reactions and satisfaction rates. 


Colorado Challenge: 


To continue to encourage campuses to address 
penetration rates through common methodologies. 
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